Approximate inference for state-space models

This thesis is concerned with state estimation in partially observed diffusion processes with discrete time observations. This problem can be solved exactly in a Bayesian framework, up to a set of generally intractable stochastic partial differential equations. Numerous approximate inference methods exist to tackle the problem in a practical way. This thesis introduces a novel deterministic approach that can capture non normal properties of the exact Bayesian solution. The variational approach to approximate inference has a natural formulation for partially observed diffusion processes. In the variational framework, the exact Bayesian solution is the optimal variational solution and, as a consequence, all variational approximations have a universal ordering in terms of optimality. The new approach generalises the current variational Gaussian process approximation algorithm, and therefore provides a method for obtaining super optimal algorithms in relation to the current state-of-the-art variational methods. Every diffusion process is composed of a drift component and a diffusion component. To obtain a variational formulation, the diffusion component must be fixed. Subsequently, the exact Bayesian solution and all variational approximations are characterised by their drift component. To use a particular class of drift, the variational formulation requires a closed form for the family of marginal densities generated by diffusion processes with drift components from the aforementioned class. This requirement in general cannot be met. In this thesis, it is shown how this coupling can be weakened, allowing for more flexible relations between the variational drift and the variational approximations of the marginal densities of the true posterior process. Based on this revelation, a selection of novel variational drift components are proposed.